Data Mining: Some examples and some directions

Many successful attempts have been made to deduce information from data. Such data might come from an explicit body of knowledge, for example from a data base. Alternatively the data might be implicit, for example the behaviour of a system. The system may itself be physical such as a helicopter or a factory, or it may be conceptual such as a financial system. The extraction of information from such data is increasingly being referred to as Data Mining or even as Data Base Mining even though finding patterns in data has been a preoccupation of scientists for thousands of years. Techniques from Artificial Intelligence such as Rule Induction, Artificial Neural Networks and Genetic Algorithms, as well as the traditional statistical clustering techniques can be used effectively on small to medium sized data sets. We illustrate the use of these techniques by giving examples of applications ranging from the prediction of strain on helicopter parts through demand forecasting to applications in shop floor scheduling and the prediction of foreign exchange rates. Despite the undoubted success of these methods, problems arise when the size of the data set becomes very large. These problems of speed and space that could previously be ignored when dealing with small data sets become more important and may even dominate once the set becomes large. Another important issue is the understandability of the model produced. Twentieth century humans seem to be much less able to accept successful but unexplained models than their ancestors. In this article we look at the developments of our ideas in Data Base Mining influenced by the rapid expansion in the quantity of data available to companies.